Autonomous Reinforcement Learning with Experience Replay for Humanoid Gait Optimization
نویسنده
چکیده
This paper demonstrates application of Reinforcement Learning to optimization of control of a complex system in realistic setting that requires efficiency and autonomy of the learning algorithm. Namely, Actor-Critic with experience replay (which addresses efficiency), and the Fixed Point method for step-size estimation (which addresses autonomy) is applied here to approximately optimize humanoid robot gait. With complex dynamics and tens of continuous state and action variables, humanoid gait optimization represents a challenge for analytical synthesis of control. The presented algorithm learns a nimble gait within 80 minutes of training. c © 2012 The Authors. Published by Elsevier B.V. Selection and/or peer-review under responsibility of the Program Committee of INNS-WC 2012.
منابع مشابه
Robo-Erectus: a low-cost autonomous humanoid soccer robot
The humanoid soccer robot league is a new international initiative to foster robotics and AI technologies using soccer games [1]. This paper provides a brief description of a low-cost autonomous humanoid soccer robot called Robo-Erectus (RE), which has been developed in the Center for Advanced Robotics and Intelligent Control (ARICC) at Singapore Polytechnic since 2001. To develop a low-cost hu...
متن کاملReal-world reinforcement learning for autonomous humanoid robot docking
Reinforcement learning (RL) is a biologically supported learning paradigm, which allows an agent to learn through experience acquired by interaction with its environment. Its potential to learn complex action sequences has been proven for a variety of problems, such as navigation tasks. However, the interactive randomized exploration of the state space, common in reinforcement learning, makes i...
متن کاملOn Global Optimization of Walking Gaits for the Compliant Humanoid Robot, COMAN Using Reinforcement Learning
In ZMP trajectory generation using simple models, often a considerable amount of trials and errors are involved to obtain locally stable gaits by manually tuning the gait parameters. In this paper a 15 degrees of Freedom dynamic model of a compliant humanoid robot is used, combined with reinforcement learning to perform global search in the parameter space to produce stable gaits. It is shown t...
متن کاملDesign of an Autonomous Fast-Walking Humanoid Robot
The design considerations for a small, relatively fast walking, autonomous humanoid robot are presented. The robot must be energy efficient but also produce sufficient torque to reach greater speeds. On the basis of previous investigations into gait optimization for multilegged systems, dynamical modeling and nonlinear optimization tools are used for design optimization and choosing the motor s...
متن کاملLearning Fast Bipedal Locomotion Master’s Thesis
In this thesis, I present a method to optimize the clock-driven, periodical walking pattern of a humanoid robot for forward speed using metaheuristics. I start from a hand-tuned open-loop gait and enhance it with two feedback control mechanisms. First, I employ a P-controller to regulate the foot angle to reduce angular velocity of the robot’s body. The angular velocity is measured by a gyrosco...
متن کامل